In various electronics industries, particularly the storage industry, PCI-E has become a popular communications interface for connecting input/output (I/O) devices to root complexes. In a typical PCI-E system, a root complex connects a processor system (with one or more CPUs and memory) to various endpoints (e.g., I/O devices) over point-to-point serial connections, referred to as links. Each link has a width determined by its number of lanes. Each lane contains two pairs of wires: one pair for receiving and the other for transmitting signals. Commonly used link widths include four lanes (×4), eight lanes (×8), and 16 lanes (×16).
To produce flexible, manufacturable, and serviceable products, some electronics systems (e.g., embedded storage arrays) implement the root complex and I/O devices on separate pluggable modules (i.e., as CPU modules and I/O modules, respectively). A given CPU module typically communicates with a set of I/O modules. The number of I/O modules in the set can vary from product to product, depending upon, for example, mechanical restrictions such as the physical sizes of the various modules and of the enclosure, and upon the particular market (e.g., high-end or low-end) for which the product is designed.
The electrical matching between a CPU module and a set of I/O modules presents various problems. For one, the root complex of the CPU module supports a fixed number of PCI-E lanes, which need routing to the slot connectors of the set of I/O modules. If a perfect multiple of lanes is not available, the I/O module slot connectors may need to receive different numbers of lanes. Further, the components on different I/O modules (e.g., Fibre Channel, Ethernet, SAS (serial attached SCSI)) may support different lane widths (e.g., ×4, ×8) regardless of the lane-width capabilities routed to a given I/O module slot connector. PCI-E switches can mitigate this connectivity imbalance, but generally affect the performance of the I/O module slots to which such PCI-E switches attach. In addition, the various CPU modules available in the market have varying capabilities (e.g., different speeds).
Standard and custom PCI-E chipsets presently incorporate automated features that relieve some of the above-described problems. Link-width negotiation allows two interconnected modules to train to the lowest common link width. Speed negotiation enables training to the lowest speed that both modules on the link can support. In addition, some root complexes support auto-bifurcation: the root complex is able to differentiate between a single large-width I/O module and multiple smaller-width I/O modules, and train accordingly. Notwithstanding, some root complexes cannot bifurcate automatically and require pre-configuration if the root complex is to bifurcate during training. Despite these automated features, however, limitations remain that impede the ability to flexibly dedicate a specific number of lanes to each I/O module slot connector in order to achieve fully subscribed connectivity across a broad range and multiple generations of CPU and I/O modules.